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ABSTRACT 


This  report  records  the  content  of  a  presentation  on 
options  for  Search  and  Rescue  performance  indicators  that 
could  be  utilized  by  the  Department  of  National  Defence.  The 
work  was  conducted  as  component  of  a  research  project  to 
evaluate  the  SAR  data  collection  system  and  its  capability  to 
support  SAR  performance  assessment.  Delays  in  the  project  have 
created  the  potential  for  the  loss  of  this  information.  This 
report  was  prepared  to  preserve  this  material  in  an  easily 
accessible  form. 


i 


P500614.PDF  [Page:  8  of  66] 


P500614.PDF  [Page:  9  of  66] 


SEARCH  AND  RESCUE  PERFORMANCE  INDICATORS 
OPTIONS  FOR  CONSIDERATION 


BACKGROUND 


1.  In  1993,  the  Air  Operation  Research  Team,  then  known  as 
the  Directorate  of  Air  Operational  Research  (DAOR) ,  undertook 
a  study  to  identify  performance  indicators  for  Search  and 
Rescue.  This  study  was  to  identify  indicators  that  could  be 
utilized  by  the  Department  of  National  Defence  (DND)  to 
present  to  the  Government  and  the  general  public  DND 
performance  in  executing  its  SAR  mandate .  The  performance 
indicators  were  also  to  be  useful  to  DND  to  evaluate  its  SAR 
performance,  identify  developing  trends,  and  take  action  where 
and  when  required. 

2.  Any  performance  indicators  proposed,  had  to  be  viable  in 
the  sense  that  any  data  required  to  define  the  indicator  had 
to  be  available  or  easily  collected  by  DND.  Many  possible 
performance  indicators  could  not  meet  this  criterion. 

3.  DAOR  identified  many  possible  performance  indicators  that 
could  be  of  interest  and  use  to  several  levels  of  DND  SAR 
managers.  These  indicators  were  primarily  developed  from  data 
collected  and  stored  in  the  SAR  database  SARIS,  and  from  SAR 
evaluation  models  developed  in  DAOR.  These  options  for  SAR 
performance  indicators  were  presented  to  DND  SAR  managers  in 
a  k^isfing  in  1993.  The  contents  of  the  briefing  were  not 
formally  documented  beyond  the  briefing  notes  provided  at  the 
presentation . 

4.  In  1993  DND  was  in  the  midst  of  switching  to  a  new  SAR 
data  collection  and  archival  system.  It  was  intended  that  the 
work  on  SAR  performance  indicators  would  continue  with  the 
development  of  the  new  SAR  database.  At  the  time  of  this 
wr-*-ting  the  transfer  to  the  new  SAR  database  system  is  not  yet 
complete.  The  air  operational  research  group  has  a  standing 
project  to  evaluate  the  new  SAR  database  system  when  the 
system  is  completed  and  access  available. 
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5.  A  component  of  the  SAR  database  evaluation  will  be  an 
assessment  of  the  capability  of  the  database  system  to  support 
DND  SAR  performance  appraisal.  The  SAR  performance  indicator 
options  identified  in  the  1993  briefing  are  proposed  as  the 
basis  for  this  component  of  the  database  evaluation. 

6.  With  the  impending  transfer  of  personnel,  the  delay  in 
the  completion  of  the  SAR  database  system,  and  the  delay  in 
the  initiation  of  the  database  evaluation,  there  is  a  risk  of 
possible  loss  information  and  continuity  in  this  project.  For 
these  reasons,  the  content  of  the  1993  briefing  on  SAR 
performance  indicator  options  has  been  formally  recorded  in 
this  report.  What  follows  is  the  text  and  slides  used  in  the 
aforementioned  briefing. 


SLIDE  ONE 


7 .  The  following  text  represents  a  few  brief  notes  to 
explain  the  contents  of  the  attached  slides.  This  package  is 
intended  to  provide  a  concise  description  of  DAOR's  proposal 
for  DND  SAR  performance  indicators  and  will  serve  as  a  basis 
to  promote  thought  and  comment. 


SLIDE  TWO 


8.  DAOR  undertook  this  activity  in  response  to  a  study 
request  from  the  former  National  Defence  Headquaters  (NDHQ) 
Directorate  of  Air  Operations  and  Training  (DAOT) .  The  study 
essentially  requested  DAOR  to  identify  Search  and  Rescue 
performance  indicators  that  could  be  used  to  inform  the 
general  public  of  the  level  of  service  provided  by  DND  and 
could  be  used  by  the  Department  to  quantify  and  evaluate  the 
level  of  service  provided  by  the  Department  to  Search  and 
Rescue.  Presumably  these  performance  indicators  would/ should 
also  be  utilized  to  monitor  performance  levels  and  identify 
when  changes  were  occurring  in  the  system. 


i 


i 
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SLIDE  THREE 


9.  The  overriding  criteria  used  by  DAOR  to  judge  the 
appropriateness  of  possible  indicators  was  that  the 
indicators  should  be  evaluating  the  effect  of 
factors  over  which  DND  has  some  control. 


SLIDE  FOUR 


10.  It  is  DAOR ' s  contention  that  the  SAR  performance 
indicators  should  fulfil  several  functions.  The  indicators 
should  quantify  the  current  level  of  DND  SAR  performance.  They 
should  permit  comparison  with  past  levels  of  service  to  allow 
the  determination  of  the  existence  of  trends  and  changes  that 
may  be  occurring  in  the  overall  SAR  system,  be  it  the  rescue 
system  or  the  nature  of  the  SAR  incidents.  Also,  the 
performance  indicators  should  be  useful  towards  the  adoption 
of  SAR  standards  of  performance,  should  the  identification  of 
performance  standards  be  desired.  Finally,  the  performance 
indicators  should  support  and  promote  planning  and  decision¬ 
making  for  DND  SAR  operations  and  system  development. 


SLIDE  FIVE 


11.  The  performance  indicators  proposed  by  DAOR  can  be 
divided  into  two  classifications:  Reaction  Capability  and 
Response  Capability.  Furthermore,  the  Reaction  Capability 
indicators  can  be  divided  into  basic  measures  and  composite 
(system)  measures,  while  the  Response  Capability  indicators 
can  be  grouped  under  coverage,  relative  response  capability 
(comparison  with  past  SAR  incidents)  ,  and  intensity  (magnitude 
of  demands  on  the  DND  SAR  system) .  The  performance  indicators 
were  chosen  from  the  perspective  that  data  to  derive  the 
indicators  must  be  readily  available. 
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SLIDE  SIX 


12.  The  SAR  incident  time  line  as  identified  by  the  Cross 
Commission  Report  was  used  as  the  basis  for  the  formulation  of 
the  reaction  capability  performance  indicators.  The  incident 
time  line  divides  the  total  SAR  incident  into  a  series  of 
actions  that  occur.  At  the  beginning,  the  incident  occurs 
(i.e.,  a  plane  crashes).  This  is  followed  some  responsible 
agency  being  alerted  to  the  possibility  of  an  incident. 
Eventually,  a  Rescue  Coordination  Centre  is  notified.  There  it 
is  decided  if  the  incident  is  valid  and  warrants  the  launch  of 
a  resource.  If  so,  a  resource  is  tasked,  launched,  transits  to 
the  site,  searches  and  hopefully  locates  the  object  of  the 
incident.  Once  the  object  is  located,  personnel  and  equipment 
are  deployed  to  provide  assistance. 


1 


SLIDE  SEVEN 


13 .  As  DND  responsibility  for  a  SAR  incident  usually  begins 
with  the  notification  of  the  Rescue  Coordination  Centre  (RCC) , 
this  was  used  as  the  starting  point  for  the  specification  of 
reaction  capability  indicators.  The  "basic  measures"  were 
formulated  to  isolate  performance  levels  in  each  of  the  main 
actions  that  follow  RCC  notification  until  the  onset  of 
assistance  being  provided.  These  indicators  include: 

Mobilization  Time  -  the  time  from  RCC  notification  until 

the  first  DND  resource  launches  to 
proceed  to  the  incident  site; 

-  the  time  from  the  launch  of  the  first 
resource  to  its  arrival  on-scene; 

-  the  time  from  the  arrival  of  the 
first  resource  on-scene  until  the  SAR 
object  is  located;  and, 

-  the  time  interval  between  the 
location  of  the  SAR  object  to  the 
start  of  assistance  being  provided. 


Transit  Time 
Search  Time 

Deploy  Time 
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SLIDE  EIGHT 


14.  The  "composite  measures"  capture  the  accumulation  effect 
of  several  of  the  basic  actions  taking  place  in  the  resolution 
of  the  SAR  incident.  These  include: 

System  Action  Time  -  the  time  from  RCC  notification 

until  the  first  resource  arrives 
on-scene; 

System  Location  Time  -  the  time  from  RCC  notification  to 

the  SAR  object  being  located;  and. 

System  Response  Time  -  the  time  from  RCC  notification  to 

the  start  of  assistance  being 
provided. 


SLIDE  NINE 


15.  The  Response  Capability  coverage  indicators  would  consist 
of  a  map  showing  the  total  area  of  Canadian  SAR  responsibility 
with  an  overlay  (s)  displaying  the  total  area  that  can  be 
reached  by  DND  resources  considering  platform  limitations, 
flight  regulations,  and  refuelling  location  availability.  A 
second  coverage  indicator  would  consist  of  a  map  showing  time 
contours  indicating  the  ideal  transit  time  to  arrive  at  the 

contour  locations  considering  the  same  factors  as  for  the 
coverage  map. 

16.  For  Relative  Response  Capability,  ideal  transit  and 
transport  effectiveness  were  chosen.  These  indicators  will  be 
derived  from  past  incident  occurrences.  Specifically,  the 
proportion  of  SAR  incidents  that  are  within  resource  rescue 
range  from  the  nearest  refuelling  location  and  the  proportion 
of  incidents  where  all  the  lives  at  risk  could  be  transported 
in  a  single  sortie  will  be  determined. 
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17.  The  final  class  of  proposed  Response  Capability 
indicators  is  Intensity  that  represents  the  magnitude  of  the 
demands  placed  upon  the  DND  SAR  system.  Three  performance 
indicators  for  this  class  are  proposed: 

RCC  Case  Load  -  the  total  number  of  SAR  incidents  that 

the  RCC  dealt  with  during  the  course  of 
the  year; 

Resource  Demand  -  the  number  of  SAR  incidents  during  the 

year  which  involved  DND  primary  SAR 
resources ;  and , 

Lives  at  Risk  -  the  total  number  of  lives  that  were  at 

risk  in  the  SAR  incidents  where  DND 
primary  SAR  resources  were  involved. 


SLIDE  11 


I 

I 

i 
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18 .  The  reaction  capability  performance  indicators  were  ' 

chosen  because  reaction  time  is  critical  while  saving  lives.  . 

As  such,  reaction  time  should  be  a  factor  utilized  in  1 

indicators  employed  to  identify  the  performance  of  the  SAR 
system.  As  well,  reaction  time  has  been  used  prominently  in  ] 

past  evaluations  of  the  National  and  DND  SAR  programmes  and  \ 

one  can  expect  this  factor  to  be  used  in  any  future 
evaluations.  Any  performance  monitoring  system  that  did  not  ! 

utilize  reaction  time  in  some  capacity,  is  likely  to  come 

under  severe  criticism. 

19.  Coverage  and  the  other  response  capability  indicators  are 
important  for  the  identification  and  elimination  of  gaps  and 
weak  areas  in  the  rescue  system  and  the  demands  placed  upon 
it. 

.* 

20.  Finally,  the  performance  indicators  proposed  should  be  ^ 

able  to  promote  the  early  identification  of  changes  which  may 

be  taking  place  in  the  system.  The  determination  of  the 


i 
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specific  factors  which  are  causing  the  changes  will  probably 
require  separate  detailed  and  directed  studies,  but  it  will  be 
the  performance  indicators  that  will  highlight  the  requirement 
for  these  studies.  The  performance  indicators  should  point  out 
the  need  for  assessments  and  decisions  to  address  changes 
occurring  in  the  SAR  system. 


SLIDES  12.  13.  &  14 


21.  The  strength  of  performance  indicators  is  directly 
f-o  the  number  of  factors  whose  impact  on  system 
effectiveness  is  directly  or  indirectly  observable  through 
monitoring  the  performance  indicators.  These  slides  list  the 
plethora  of  factors  that  have  a  significant  impact  on  the 
effectiveness  of  DND  to  carry  out  its  SAR  mandate.  The 
composite  effect  of  the  factors  and  the  effect  of  changes  in 
the  factors  should  be  observable  through  regular  examination 
of  the  proposed  performance  indicators.  Monitoring  the 
proposed  performance  indicators  should  foster  the  early 
discovery  of  changes  occurring  and  allow  reaction  in  a  timely 
fashion. 


SLIDES  15.  16.  &  17 


22.  The  following  slides  represent  examples  of  what  is 
intended  for  the  SAR  performance  indicators  and  what  form  they 
could  take.  Although  some  slides  are  derived  from  current  SAR 
data,  they  are  intended  as  examples  only,  and  should  not  be 
interpreted  as  an  analysis  of  current  DND  SAR  performance. 

23.  Slides  15,  16,  and  17  contain  the  "raw"  statistics  for 
the  proposed  reaction  performance  indicators,  in  this  case  for 
the  year  1988.  The  prime  characteristics  of  the  data 
distribution  for  the  performance  indicator  is  shown  by  the 
mean  time,  the  standard  deviation  (STD) ,  and  the  minimum  and 
maximum  values.  (All  times  are  in  minutes.)  The  number  of 
acceptable  data  points  is  shown  (DATA) .  The  general  shape  of 
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the  frequency  distribution  can  be  deduced  from  a  listing  of 
decile  values.  Each  decile  interval  indicates  the  time 
interval  within  which  10  percent  of  the  incidents  (data 
values)  occurred.  For  example,  for  mobilization  in  Slide  15, 
a  DND  resource  was  launched  within  12  minutes  in  10  percent  of 
the  SAR  cases,  while  a  resource  was  launched  within  26  minutes 
in  20  percent  of  the  SAR  incidents,  etc. 

24.  While  these  data  contain  much  information,  few  insights 
are  easily  obtained.  It  is  suggested  that  such  data  be 
provided  in  an  annex  to  the  SAR  performance  (indicator) 
report. 


SLIDES  18-24 


25.  These  slides  contain  charts  of  the  plotted  values  of  the 
mean,  30,  60  and  90  percentiles  of  the  reaction  performance 
indicators  for  the  years  1988-1991.  Also,  shown  on  the  chart 
are  linear  regression  trend  lines.  The  lines  do  not 
necessarily  indicate  that  a  trend  is  present;  statistical 
analysis  of  the  data  would  be  required  to  confirm  or  deny  the 
presence  of  a  trend.  However,  the  trend  lines  present  a  useful 
(but  crude)  indication  of  the  likelihood  of  a  trend  and  its 
magnitude  and  indicate  where  further  statistical  analysis  is 
justified. 

26.  Performance  indicators  plotted  in  this  fashion  (  and 
supported  with  statistical  analysis)  permit  conclusions  to  be 
made  regarding  changes  which  may  be  evolving  in  the  SAR 
system.  For  example,  if  it  is  assumed  that  the  trends  plotted 
in  the  chart  for  mobilization  (Slide  18)  have  been  shown  to  be 
statistically  valid,  it  can  be  noted  that  the  mobilization 
time  for  the  30  and  60  percentiles  are  effectively  constant. 
The  mobilization  times  for  the  90  percentile  are 
(sporadically)  decreasing  from  which  it  can  be  concluded  that 
the  number  of  incidents  with  very  long  mobilization  times  are 
decreasing.  But  the  average  (mean)  mobilization  time  is  slowly 
but  consistently  increasing. 
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SLIDES  25-? 1 


27.  These  slides  display  charts  where  the  percentiles  for  the 
reaction  performance  indicators  have  been  plotted.  The  100 
percentile  has  been  omitted  to  avoid  making  the  vertical  scale 
excessive  to  accommodate  the  extreme  last  point  in  the 
distribution  and  obscuring  most  of  the  detail  in  the  rest  of 
the  distribution.  Charts  such  as  these  permit  patterns  and 
emerging  trends  to  be  discerned. 

28.  For  example,  in  Slide  25  Mobilization  Trends,  one  can  see 
that  the  distributions  for  the  years  1988  to  1991  are 
approximately  identical.  One  could  conclude  that  there  has 
been  no  increase  or  degradation  in  mobilization  performance 
during  this  time  period.  Examination  of  the  following  slide  of 
Transit  Trends  yields  the  conclusion  that  transit  times  in  the 
last  two  years  have  generally  increased  by  10-20  minutes.  It 
must  be  remembered  that  nothing  can  be  concluded  about  what 
factor  may  have  caused  this  slight  increase.  If  it  is  believed 
that  a  trend  such  as  this  is  serious,  a  detailed  study  should 
be  initiated  to  determine  the  validity  and  possible  cause  of 
the  observed  trend. 


SLIDE  32 


29.  This  slide  is  representative  of  a  coverage  overlay  map  to 
address  one  of  the  response  capability  indicators.  This  slide 
is  presented  to  provide  a  visual  indication  of  type  of  display 
DAOR  has  in  mind  for  this  performance  indicator.  This 
particular  map  was  drawn  from  some  earlier  work  DAOR  performed 
for  the  New  SAR  Helicopter  (NSH)  requirements  study  and  should 
not  be  viewed  as  an  "exact"  proposal.  DAOR  is  currently 
working  towards  developing  a  procedure  that  will  produce  a 
coverage  overlay  map  in  perspective  and  detail  best  to 
facilitate  its  interpretation. 

30.  It  should  be  noted  that  the  process  to  determine  the 
coverage  bounds  relies  on  a  program  that  considers  the 
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distribution  of  available  refuelling  locations,  the 
limitations  of  the  airframe,  the  locations  of  the  operating 
bases,  and  the  flight  regulations  applied.  The  program 
determines  the  best  route  for  the  aircraft  to  follow  and  which 
refuelling  points  to  use  to  arrive  at  its  destination.  The 
program  also  determines  the  limit  of  travel  for  the  aircraft 
if  such  a  limit  exists. 


SLIDE  33 


1 

1 

I 

i 

i 

\ 


31.  Like  the  previous  slide,  this  slide  is  intended  to  be 
representative  of  the  general  form  of  the  answer  to  the  Time 
Contour  performance  indicator.  Again,  this  particular  chart 
was  drawn  from  earlier  work  for  the  NSH.  DAOR  is  currently 
examining  how  best  to  represent  the  information  on  the  map. 
However,  the  intent  is  to  display  the  ideal  time  taken  by  an 
aircraft  to  reach  a  given  location  and  to  show  which  locations 
are  beyond  the  reach  of  the  airframe.  The  same  considerations 
apply  to  determine  the  optimum  route  as  was  mentioned  for  the 
previous  slide,  namely,  airframe  limitations,  refuelling 
locations,  and  flight  regulations. 


SLIDE  34 


32.  A  plot  similar  to  the  one  shown  here  would  be  used  to 
display  the  proportion  of  past  incidents  that  were  within  the 
aircraft  rescue  range  from  the  nearest  refuelling  point.  This 
is  calculated  for  all  past  incidents,  not  just  those  to  which 
DND  responded.  This  performance  indicator  provides  an 
indication  of  the  potential  of  DND  resources  to  assist  in  a 
SAR  incident.  By  examining  for  possible  trends,  one  can 
determine  if  the  geographic  distribution  of  SAR  incidents  is 
changing  such  that  the  transit  capabilities  of  DND  aircraft 
are  exceeded  more  frequently  (indicating  the  requirement  for 
modif ications/updates  or  procurement) .  It  should  be  noted  that 
the  data  in  this  chart  are  totally  fictitious. 


I 

I 

J 

I 

\ 
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SLIDE  35 


33.  This  chart  displays  the  frequency  of  occurrence  of  the 
various  levels  of  people  on  board  (POB)  aircraft  and  vessels 
in  SAR  incidents.  This  can  be  interpreted  as  potential  lives 
at  risk.  The  distributions  for  the  four  years  shown  indicate 
that  little  or  no  change  is  taking  place  over  the  years.  The 
vast  majority  of  SAR  incidents  involve  fewer  than  10  people. 


SLIDE  3  6 


34.  This  chart  depicts  the  transport  capability  performance 
indicator  for  the  current  SAR  helicopter.  It  identifies  the 
helicopter's  capability  to  transport  all  the  lives  at  risk  in 
an  incident  in  one  sortie.  It  can  be  seen  that  the  current 
helicopter  can  transport  all  the  lives  at  risk  in  one  sortie 
m  approximately  99  percent  of  all  SAR  incidents  and  that  this 
transport  performance  is  consistent  year  after  year. 


SLIDE  37 


35.  The  following  charts  display  typical  results  for  the 
performance  indicators  quantifying  Intensity  response 
capability.  This  first  slide  identifies  the  annual  load  of  SAR 
cases  that  the  DND-operated  rescue  coordination  centres  must 
handle.  From  the  chart,  it  can  be  seen  that  1988  was  a  more 
intense  year,  while  the  RCC  case  load  has  been  relatively 
constant  from  1989  to  1991.  The  chart  also  displays  how 
quantities  of  the  various  incident  classes  may  be  changing. 


SLIDE  38 


36.  The  annual  incident  load  that  DND  primary  SAR  resources 
respond  to  is  shown  here.  Again,  the  case  load  is  broken  down 
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by  incident  type.  There  appears  to  be  a  slight  trend  in 
decreasing  case  load.  Statistical  analysis  would  be  required 
to  confirm  this  and  determine  the  confidence  level.  This 
display  is  proposed  for  the  Primary  Resource  Demand 
performance  indicator. 

SLIDE  39 

37.  The  values  of  the  proposed  lives  at  risk  performance 
indicator  are  shown  in  this  slide.  These  are  the  total  number 
of  lives  at  risk  in  all  the  SAR  incidents  to  which  DND  primary 
SAR  resources  responded.  This  indicator  provides  an  impression 
of  the  level  of  responsibility  and  importance  of  DND  SAR 
resource  activities.  From  the  slide  there  appears  to  be  a 
trend  towards  increasing  lives  at  risk. 


SLIDE  40 


38.  Statistical  analysis  has  been  mentioned  several  times  in 
this  briefing.  It  is  required  to  support  what  may  visually  . 

appear  to  be  emerging  trends  in  the  data  for  the  performance  j 

indicators.  It  was  felt  worthwhile  to  provide  an  example  of 
the  sort  of  statistical  analysis  that  could  be  performed  to 
validate  a  possible  trend.  For  this  example  we  will  use  the  ' 

data  of  the  previous  chart  for  lives  at  risk.  From  the  chart, 
it  can  be  seen  that  there  appears  to  be  a  trend  of  increasing 
level  of  lives  at  risk.  The  first  statistical  calculation  that 
can  be  done  is  to  determine  the  correlation  coefficient 
between  the  level  of  lives  at  risk  and  the  year.  This  value 
will  indicate  how  well  one  can  predict  the  lives  at  risk  given 
the  value  for  the  year  under  consideration.  For  these  data  the 
correlation  coefficient  works  out  to  be  0.961.  For  four  points 
a  value  above  0.90  indicates  that  a  conclusion  that  a 
correlation  exists  is  justifiable.  In  this  case,  it  could  be 
concluded  that  there  is  a  correlation  in  the  data.  The  next 
step  is  to  estimate  the  confidence  level  that  can  be 
associated  with  the  hypothesis  that  the  correlation  is  not 
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produced  through  random  effects.  For  this,  a  Pitman 
statistical  test  can  be  applied  to  the  data.  The  Pitman  test 
is  a  non-parametric  test  to  estimate  the  probability  that  the 
data  are  independent.  In  the  case  at  hand,  the  two-sided 
significance  level  is  0.0797,  which  is  to  say  that  there  is  a 
7 . 97  percent  chance  that  the  observed  trend  is  produced  by 
random  chance.  This  can  be  interpreted  as  stating  that  there 
— ^Y-  strong  evidence  that  the  trend  is  valid  (nothing  is 
ever  certain  in  statistics)  .  From  all  of  this  one  could  assume 
that  it  is  reasonably  safe  to  state  that  there  is  a  trend 
towards  an  increasing  annual  total  of  lives  at  risk  for  SAR 
incidents  to  which  DND  resources  respond. 


SLIDE  41 


39.  In  conclusion,  it  is  felt  that  the  Search  and  Rescue 
performance  indicators  proposed  provide  a  broadly  based 
evaluation  of  the  performance  of  the  DND  SAR  system.  They  can 
be  used  to  inform  the  public  of  DND' s  contribution  to  SAR  and 
the  level  of  performance  that  can  be  expected.  The  indicators 
provide  a  means  to  identify,  in  a  timely  manner,  when 
significant  changes  to  the  system  are  occurring  and  additional 
detailed  investigation  is  required.  The  proposed  performance 

indicators  provide  support  for  SAR  planning  and  decision¬ 
making. 
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SAR  PERFORMANCE  INDICATORS 
STUDY  REOMFRT 


"...  identify  specific  performance 
indicators  that  can  be  used  to 
measure  the  level  of  service  provided 
to  the  Canadian  public.  These 
performance  indicators  could  also  be 
used  to  evaluate  the  delivery  of  SAR 
services.  ” 


...  identify  appropriate  performance 
indicators  related  to  the  DND  SAR 
programme  with  a  view  to  quantifying 
the  baseline  level  of  service  provided 
at  the  current  time.  The  study  should 
include  all  aspects  of  the  Department’s 
air  and  marine  mandate.  ” 


^3.8.5-9-15  ( DAOT  5)  dated  5  .Inn  Q? 
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Search  and  Rescue 
Performance  Indicators 


Selection  Criteria: 

Only  measures  of  effectiveness 
on  which  DND  could  have  some 
impact  would  be  appropriate. 


SLIDE  3 
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SAR  PERFORMANCE  INDICATORS 
UTIL1TY/REQU1REMFNT 


•  identify  current  level  of  performance 

•  support  comparison  with  past  performance 

•  permit  analysis  for  early  warning  of 
changes  and  trends 

•  allow  specification  of/comparison  with 
SAR  goals  and  standards 

•  provide  support  for  SAR  planning  and 
decision-making 
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Search  and  Rescue 


Performance  Indicators 


1.  Reaction  Caoabilit 


-  Basic  Measures 

-  Composite  Measures 


2.  Response  Caoabilit 


-  Coverage 

-  Relative  Response 
Capability 

-  Intensity 
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SAR  INCIDENT  TIME  LINE 
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SAR  Performance  Indicators 


Reaction  Cacabilit 


Basic  Measures 


1.  SAR  Mobilization  Time 


Time  from  RCC  notification  to  time 
first  SAR  resource  departs. 

Decision  time  (T4)  +  Reaction  Time  (T5) 
Addresses  adequacy  of  RCC  procedures 
&  support  equipment  and  MOB  alert 
postures. 

2.  Resource  Transit  Time  (T6) 


Addresses  adequacy  of  resource  speed, 
endurance,  and  base  location. 

3.  SAR  Search  Time  (T 7) 


For  incidents  where  object  is  found. 
Addresses  adequacy  of  resources  search 
capability. 

4.  Rescue  Deploy  Time  (T8) 


Addresses  adequacy  of  resource  capability 
to  deploy  SARTechs  and  equipment. 


Based  on  Cross  Report  "Time  Line’ 
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SAR  Performance  Indicators 
Reaction  Capability 

Composite  Measures 

5.  SAR  System  Action  Time 
Mobilization  time  +  transit  time 
(T4+T5+T6) 

Indicator  for  time  to  arrive  on¬ 
scene  from  RCC  notification. 

6.  SAR  System  Location  Time 

Sar  System  Action  Time  +  Search 
Time  (T4+T5+T6+T7) 

Indicator  for  time  to  locate 
object  from  RCC  notification. 

7.  SAR  System  Response  Time 
System  Location  Time  +  Deploy  Time 
(T4+T5+T6+T7+T8) 

Indicator  for  time  to  provide  medical 
assistance  from  RCC  notification. 


Based  on  Cross  Report  "Time  Line” 


SLIDE  8 
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SAR  Performance  Indicators 
Response  Capability 


Coverage 

1.  Coverage  Overlay  Maos 

Maps  showing  ail  areas  which  can 
reached  by  SAR  resources. 

2.  Time  Contour  Maos 

Maps  displaying  contours  for 
resource  transit  time  from  base. 


Relative  Response  Capability 

3.  Transit  Capability 
Proportion  of  past  SAR  incidents 
within  resource  rescue  range  from 
nearest  refueling  location. 

4.  Transport  Capability 
Proportion  of  past  SAR  incidents 
where  total  lives  at  risk  could  be 
accommodated  by  one  sortie. 
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SAR  Performance  Indicators 
Response  Capability 

Intensity 

1.  RCC  Case  Load 

Annual  number  of  SAR  incidents  which 
RCC  handled. 

2.  Primary  Resource  Demand 

Annual  number  of  incidents  to  which 
DND  primary  SAR  resources  responded. 

3.  Lives  at  Risk 

Number  of  lives  at  risk  in  incidents 
where  DND  primary  SAR  resources 
were  involved. 
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SAR  Performance  Indicators 
Selection  Rationale 


1.  Reaction  time  is  critical  in  the 
SAR  process  to  save  lives. 

2.  Reaction  capability  has  been 
utilized  prominently  in  past 
evaluations  of  the  SAR  system. 
(Cross  Commission  Report,  NDHQ 
Program  Evaluation  El/90,  etc) 

3.  Important  to  be  aware  of  gaps 
and  weak  areas  in  SAR  coverage 
and  limits  of  resource  response 
capability. 

4.  Must  be  able  to  identify  when 
changes  are  beginning  to  occur 
in  the  SAR  system. 

5.  Must  have  a  sound  foundation  for 
decision  making  and  planning. 
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SAR  Performance  Indicators 
Inherent  Factors 


Performance  Indicators  Factors 


Mobilization  Time 


Transit  Time 


Search  Time 


Deploy  Time 


~  incident  validation  procedure 

-  resource  readiness 

~  preparation  and  launch  time 

-  resource  availability 

-  resource  speed  &  endurance 

-  case  geographic  distribution 

-  base  locations 

-  refueling  locations 

-  weather  capability 

-  resource  search  capability 

“  environmental  characteristics 

-  search  procedures 

-  ELT  assistance 

-  fleet  size  &  availability 

-  a/c  deployment  capability 

-  system  response  capability 

SLIDE  12 


SAR  Performance  Indicators 
Inherent  Factors  (continued-1) 


Performance  Indicators 

Composite  Reactive  Measures 

Coverage  Overlay  Map 

Time  Contour  Map 

Transit  Capability 

Transport  Capability 


Factors 

-  same  factors  as  for  basic 
reactive  measures 

-  resource  endurance 

-  base  locations 

-  refueling  locations 

i 

-  resource  speed  &  endurance  £ 

-  base  locations  1 

-  refueling  locations 

-  case  geographic  distribution 

-  resource  endurance 

-  base  &  refueling  locations 

-  case  lives-at-risk 
distribution 

-  resource  transport  capacity 
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SAR  Performance  Indicators 
Inherent  Factors  (continued-2) 

Performance  Indicators  Factors 

RCC  Case  Load  -  number  of  SAR  incidents 

by  category 

-  demand  on  RCC  operation 

Primary  Resource  Demand  -  demand  on  DND  primary 

SAR  resources 

Lives  at  Risk  -  demands  on  and  importance 

of  DND  SAR  response 


SLIDE  14 


P500614.PDF  [Page:  35  of  66] 


P500614.PDF  [Page:  36  of  66] 


a AR  PERFORMANCE  INDICATORS 

1988 


BASIC  MEASURES  OF  REACTION  PERFORMANCE 

OVERALL  SAR  MOBILIZATION  STATISTICS  FOR  THE  PERIOD  1988 


MEAN 

148 

1 

DECILE 

12 

STD 

289 

2 

DECILE 

26 

MIN 

1 

3 

DECILE 

44 

MAX 

4468 

4 

DECILE 

65 

5 

DECILE 

81 

6 

DECILE 

98 

7 

DECILE 

123 

8 

DECILE 

175 

9 

DECILE 

339 

10 

DECILE 

4468 

DATA 

775 

OVERALL  SAR  RESOURCE  TRANSIT 

STATISTICS 

FOR  THE 

PERIOD  1988  1 

MEAN 

65 

1 

DECILE 

11  * 

STD 

100 

2 

DECILE 

20 

MIN 

1 

3 

DECILE 

30  j 

MAX 

1479 

4 

DECILE 

38  1 

5 

DECILE 

45 

6 

DECILE 

57 

7 

DECILE 

70 

8 

DECILE 

88 

9 

DECILE 

120 

10 

DECILE 

1479  i 

DATA 

571 

1 

OVERALL  SAR  RESOURCE  SEARCH 

STATISTICS 

FOR  THE 

PERIOD  1988 

MEAN 

224 

1 

DECILE 

10 

STD 

540 

2 

DECILE 

18 

MIN 

1 

3 

DECILE 

28 

MAX 

5817 

4 

DECILE 

44 

5 

DECILE 

61 

6 

DECILE 

95 

7 

DECILE 

131 

8 

DECILE 

184 

9 

DECILE 

477 

10 

DECILE 

5817 

DATA 

255 
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OVERALL  SAR  RESOURCE  DEPLOYMENT  STATISTICS  FOR  THE  PERIOD  1988 


MEAN 

222 

STD 

545 

MIN 

1 

MAX 

4398 

DATA  111 


1 

DECILE 

4 

2 

DECILE 

8 

3 

DECILE 

15 

4 

DECILE 

29 

5 

DECILE 

43 

6 

DECILE 

59 

7 

DECILE 

124 

8 

DECILE 

230 

9 

DECILE 

560 

10 

DECILE 

4398 

COMPOSITE  MEASURES  OF  REACTION  PERFORM ANCF, 


SAR  SYSTEM  ACTION  TIME  STATISTICS  FOR  THE  PERIOD  lqfla 


MEAN 

189 

STD 

300 

MIN 

1 

MAX 

4567 

DATA  735 


1 

DECILE 

23 

2 

DECILE 

49 

3 

DECILE 

79 

4 

DECILE 

101 

5 

DECILE 

120 

6 

DECILE 

144 

7 

DECILE 

184 

8 

DECILE 

245 

9 

DECILE 

405 

10 

DECILE 

4567 

SAR  SYSTEM  LOCATION  TIME  STATISTICS  FOR  THE  PERIOD  1988 


MEAN 

340 

1  DECILE 

33 

STD 

609 

2  DECILE 

50 

MIN 

3 

3  DECILE 

82 

MAX 

6661 

4  DECILE 

119 

5  DECILE 

166 

6  DECILE 

230 

7  DECILE 

305 

8  DECILE 

412 

9  DECILE 

724 

10  DECILE 

6661 

DATA 

406 
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SAR  SYSTEM  LOCATION  TIME  STATISTICS  FOR  THE  PERIOD  1988 


MEAN 

188 

1 

DECILE 

23 

STD 

297 

2 

DECILE 

48 

MIN 

1 

3 

DECILE 

79 

MAX 

4567 

4 

DECILE 

101 

5 

DECILE 

120 

6 

DECILE 

145 

7 

DECILE 

181 

8 

DECILE 

245 

9 

DECILE 

399 

10 

DECILE 

4567 

DATA  7  35 


\ 
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SAR  PERFORMANCE  INDICATOR 
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SAR  PERFORMANCE  INDICATOR 

DEPLOYMENT 

TIME  (min) 
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SAR  PERFORMANCE  INDICATOR 
SYSTEM  ACTION  TIMF 

TIME  (min) 


YEAR 
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SAR  PERFORMANCE  INDICATOR 
SYSTEM  LOCATION  TIME 
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SAR  PERFORMANCE  INDICATOR 
SYSTEM  RESPONSE  TIME 


100 


0 

1988 


1989 


1990 


YEAR 


1991 
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SAR  MOBILIZATION 

(1988-1991) 

Minutes 

350 
300 
250 

200 


1988 

-+- 

1989 

1990 

-B- 

1991 

•It 


SAR  TRANSIT  TRENDS 
(1988-1991) 


G 


Minutes 
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SAR  DEPLOYMENT  TRFKIDR 

(1988-1991) 


Minutes 
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1990 

1991 
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Percentiles 
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SAR  SYSTEM  ACTION  TRENDS 
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Minutes 
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SAR  SYSTEM  RESPONSE  TRENDS 

(1988-1991) 

Minutes 


SAR  COVERAGE  OVERLAY  MAP 
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SAR  PERFORMANCE  INDICATOR 
RESPONSE  -  TRANSIT  CAPABILITY 
VFR  Flight  Regulations 


SAR  Cases  (%) 


Year 


SAR  CASES  VS  POB 
1988-1991 


SAR  CASES 


nui 


1988 

1989 

1990 


1991 


1  2  3  4  5  6  7  8  9  10  11  13  15  16  17  18  19  20  21  22  23  25  26  3133  40  50  85100 

PEOPLE  INVOLVED 
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SAR  INCIDENT  VOLUME 
1988-1991 
(TOTAL  FOR  RCC) 


Number  of  Cases  (Thousands) 


1988  1989  1990  1991 


i 

Ol 

o 

I 


Year 
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SAR  INCIDENT  VOLUME 
1988-1991 

INVOLVING  DND  PRIMARY  RESOURCES 


Number  of  Cases 
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LIVES  AT  RISK 
1988-1991 

(CASES  INVOLVING  DND  PRIMARY  RESOURCES) 


Lives  at  Risk 


Year 
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STATISTICAL  ANALYSIS 
LIVES  AT  RISK 
1988-1991 


Lives  at  Risk 


829 


735 

* 


Sample  Correlation  Coefficient  -  0.961 
Two-sided  Significance  Level  -  0.0797 


200 


O' — 

1988 


1989  1990 

Year 


1991 
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SAR  PERFORMANCE  INDICATORS 

CONCLUSIONS 


1.  Proposed  indicators  provide  the  basis 
for  a  broad  evaluation  of  DND  SAR 
performance. 


2.  Suggested  indicators  can  be  readily 
understood  and  appreciated  by  non-SAR 
specialists  (i.e.  general  public)  as 
well  as  SAR  evaluators  and  planners. 


3.  Proposed  performance  indicators 
provide  an  efficient  mechanism 
to  identify  trends  that  may 
be  occurring. 


4.  Proposed  indicators  will  provide 

sound  support  for  DND  SAR  planning 
and  decision-making. 
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